The Kullback–Leibler Divergence Rate Between Markov Sources
نویسندگان
چکیده
منابع مشابه
Rényi's divergence and entropy rates for finite alphabet Markov sources
In this work, we examine the existence and the computation of the Rényi divergence rate, lim ( ), between two time-invariant finite-alphabet Markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions and , respectively. This yields a generalization of a result of Nemetz where he assumed that the initial probabilities under and are strictly p...
متن کاملRelative Entropy Rate between a Markov Chain and Its Corresponding Hidden Markov Chain
In this paper we study the relative entropy rate between a homogeneous Markov chain and a hidden Markov chain defined by observing the output of a discrete stochastic channel whose input is the finite state space homogeneous stationary Markov chain. For this purpose, we obtain the relative entropy between two finite subsequences of above mentioned chains with the help of the definition of...
متن کاملR Enyi's Entropy Rate for Discrete Markov Sources
In this work, we extend a variable-length source coding theorem for discrete memoryless sources to ergodic time-invariant Markov sources of arbitrary order. To accomplish this extension, we establish a formula for the R enyi entropy rate lim n!1 H (n)=n. The main tool used to obtain the R enyi entropy rate result is Perron-Frobenius theory. We also examine the expression of the R enyi entropy r...
متن کاملAnalysis of optimal dynamic rate shaping of Markov-1 sources
Abstract : We discuss the behavior of the optimal solution to the dynamic rate shaping problem assuming an AR(1) source model. By analyzing the statistical and rate−distortion behavior of the different components of this minimization problem, the following key result is mathematically proven: "The set of optimal breakpoint values for any frame is invariant to the accumulated motion compensated ...
متن کاملFeed-Forward Rate Distortion Function and Markov Sources
The problem of channel coding with feedback has been extensively studied over the last 50 years. Using an ideal feedback link, the encoder knows all previously received channel outputs. Recently the duality between channel coding and rate distortion has been established in [15] and [3], leading to the problem of source coding with feed-forward. In this project we first study the general formula...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Information Theory
سال: 2004
ISSN: 0018-9448
DOI: 10.1109/tit.2004.826687